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MONDAY,  NOVEMBER  14 


1 2:00  pm  -  5:30  pm  Registration  Open  -  Constellation  Ballroom  Foyer  (2nd  Floor) 

1  JO  pm  -  1 :1 0  pm  WELCOME  REMARKS  -  Constellation  A  Ballroom 

►  Dr.  Steve  Kimmel,  Senior  Vice  President,  Alion  Science  &  Technology; 
C4ISR  Division  Chairman,  NDIA 

►  Mr.  Michael  Kutch,  Senior  Engineer,  Joint  Interoperability  Test 
Command 


1:10  pm  - 1:40  pm 


1:40  pm  - 1:50  pm 


1:50  pm  -  2:00  pm 


2:00  pm  -  2:45  pm 


KEYNOTE  ADDRESS 

Ms.  Jennifer  Walsmith,  Senior  Acquisition  Executive,  National  Security 
Agency 

FACILITATION  TEAM  INTRODUCTION 

►  Mr.  Michael  Kutch,  Senior  Engineer,  Joint  Interoperability  Test 
Command 

►  Mrs.  Suzanne  Miller,  Technical  Staff,  Software  Engineering  Institute 

AGILE  UNIVERSITY:  PRODUCT  BACKLOG  VS.  REQUIREMENTS  DOCUMENT 
IN  AN  OPEN  SPACE  FORMAT 

►  Mr.  Lee  Henson,  Chief  Agile  Enthusiast,  Davisbase  Consulting 

OPEN  SPACE  SESSION 

Discuss  current  impediments  to  Agile  in  the  workplace.  Create  a  backlog  of 
user  stories  for  workshop  development 


2:45  pm  -  3:15  pm  Afternoon  Break 


JENNIFER  S.  WALSMITH 

National  Security 
Agency  Senior 
Acquisition  Executive 

Ms.  Walsmith  has  served  as 
the  Senior  Acquisition  Exec¬ 
utive  for  the  NSA  since  Jan¬ 
uary  2006.  In  this  role,  she 
is  responsible  for  all  procure¬ 
ments  in  support  of  NS  As 
Signals  Intelligence  and 
Information  Assurance  mis¬ 
sions.  While  managing  the 
Agency’s  multi-billion  dollar 
budget,  she  has  focused  on 
balancing  acquisition  disci¬ 
pline  with  mission  agility. 
She  has  also  placed  a  strong 
focus  on  leveraging  industry 
expertise  to  maintain  NS  As 
technological  advantage. 


SUZANNE  MILLER 

Technical  Staff, 
Software  Engineering 
Institute 

Mrs.  Suzanne  (SuZ)  Miller 
is  a  senior  member  of  the 
technical  staff  at  the  Soft¬ 
ware  Engineering  Institute 
of  Carnegie  Mellon  Univer¬ 
sity,  working  in  the  Acquisi¬ 
tion  Support  Program’s  US 
Air  Force  team.  Her  most 
recent  research  has  been  fo¬ 
cused  on  understanding  the 
opportunities  and  barriers 
for  using  agile  development 
methods  in  DoD  settings. 


JAMES  BOSTON 

Software  Development 
Team  Lead,  USAFPEX 

Lt  Col  James  Boston,  USAF 
(Ret)  is  the  Software  Devel¬ 
opment  Team  Lead  work¬ 
ing  on  a  squadron  automa¬ 
tion  toolset  for  the  USAF 
called  Patriot  Excalibur 
(PEX).  PEX  is  an  extraor¬ 
dinarily  flexible  operations 
management  tool  used  by 
over  670  different  Air  Force 
squadrons  executing  a  wide 
variety  of  missions.  The 
program  has  used  Agile  de¬ 
velopment  methodologies 
for  over  eight  years,  making 
it  a  pioneer  in  the  practices 
in  DoD. 
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3:15  pm  -  3:25  pm 

AGILE  UNIVERSITY:  RELEASE  PLANNING 

►  Mr.  Lee  Henson,  Chief  Agile  Enthusiast,  Davishase  Consulting 

3:25  pm  -  3:55  pm 

SESSION  RESULTS  &  SPRINT  GROUP  SELECTIONS 

Develop  sprint  groups  to  address  impediments  identified  in  Open  Space 
Session.  Break  into  sprint  groups 

3:55  pm  -  4:05  pm 

AGILE  UNIVERSITY:  SPRINT  PLANNING 

►  Mr.  Lee  Henson,  Chief  Agile  Enthusiast,  Davishase  Consulting 

4:05  pm  -  4:45  pm 

SPRINT  GROUP  INTRODUCTION  &  PLANNING 

Meet  your  Scrum  Masters!  Plan  sprint  group  work  for  Tuesday 

4:45  pm  -  5:00  pm 

AGILE  UNIVERSITY:  MINI  RETROSPECTIVE 

►  Mr.  Lee  Henson,  Chief  Agile  Enthusiast,  Davishase  Consulting 

5:00  pm  -  5:30  pm 

AGILE  WILL  WORK  IN  DOD:  THREE  EXAMPLES 

►  Ms.  Kelly  Goshorn,  Patriot  Excalibur  (PEX)  Program  Manager,  U.S. 

Air  Force 

►  Ms.  Susi  McKee,  Operational  C2  Instrumentation  System  ( 0C2IS) 
Program  Manager,  U.S.  Air  Force 

►  Mr.  Dan  Gahafer,  Forge.mil  Program  Manager,  DoD  DISA 

5:30  pm  -  6:30  pm  Networking  Reception  Hosted  by  NDIA  -  Harborview  Room 


GREG  SHARP 

Software  Development 
Team  Lead,  USAF  PEX 

Mr.  Greg  Sharp  is  a  Soft¬ 
ware  Development  Team 
Lead  working  on  a  squad¬ 
ron  automation  toolset  for 
the  USAF,  Patriot  Excali- 
bur  (PEX).  The  product  is 
a  web-based,  line  of  busi¬ 
ness  application  used  to  au¬ 
tomate,  track  and  organize 
daily  activities  in  USAF 
squadrons.  The  program  is 
widely  recognized  as  one  of 
the  few  that  have  success¬ 
fully  implemented  Agile 
Development  within  DoD. 


SUSI  MCKEE 

OC2IS  Program 
Manager,  U.S.  Air  Force 

Susana  V.  McKee  has  25  years 
of  DoD  T&E  experience. 
She  is  the  505th  Test  and 
Evaluation  Group  Technical 
Advisor  and  currently  focuses 
on  capabilities  development 
and  integration  for  the  505th 
Command  and  Control 
Wing  (CCW)  sites  at  Hurl- 
burt  Field,  FL;  Nellis  AFB, 
NV;  and  KirtlandAFB,  NM. 
Previously,  Ms  McKee  was  a 
Technical  Advisor  at  the  505 
CCW.  That  was  preceded 
by  her  role  as  the  Director 
of  Test  for  the  605th  Test  and 
Evaluation  Squadrons  C2  and 
ISR  testing  efforts. 


KELLY  G0SH0RN 

Program  Manager, 
USAF  Flight  Scheduling 

Ms.  Goshorn  is  the  Pro¬ 
gram  Manager  for  USAF 
Flight  Scheduling  programs 
including  PEX  Squadron 
Automation  and  netcen- 
tric  Commanders  Decision 
Services  winglevel  effort. 
Goshorn  pioneered  the  use 
of  Agile  Software  Develop¬ 
ment  melded  with  CMMI 
Level  3  discipline.  Her 
programs  put  the  end-user 
in  control  of  priorities  and 
delivered  capabilities  re¬ 
sulting  in  a  highly  adaptive 
development  process  that  is 
quick  to  adapt  to  evolving 
requirements. 
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TUESDAY,  NOVEMBER  15 


7:30  am  -  5:00  pm 
7:30  am  -  8:30  am 
8:30  am  -  8:45  am 

8:45  am  -  8:55  am 

8:55  am -10:00  am 

10:00  am -10:15  am 
10:15  am -10:25  am 

10:25  am -11:00  am 
11:00  am -11:10  am 

11:10  am -11:45  am 


Registration  Open  -  Constellation  Ballroom  Foyer 
Continental  Breakfast 

WELCOME  REMARKS  &  DAY  1  REVIEW  -  Constellation  A  Ballroom 

►  Mrs.  Suzanne  Miller,  Technical  Staff,  Software  Engineering  Institute 

AGILE  UNIVERSITY:  WHAT’S  “DONE”? 

►  Mr.  Lee  Henson,  Chief  Agile  Enthusiast,  Davisbase  Consulting 

SPRINT  GROUP  WORK  —  “IDEAS  FOR  CHANGE” 

Build  integrated  solutions  that  address  the  previously  identified 
impediments.  Create  presentations  for  “ideas  for  change ” 

Morning  Break 

AGILE  UNIVERSITY:  IMPORTANCE  OF  A  “DEMO”  AND  HOW  IT  RELATES 
TO  OUR  TASK 

►  Mr.  Lee  Henson,  Chief  Agile  Enthusiast,  Davisbase  Consulting 

“DEMO”  FROM  SPRINT  GROUPS 
AGILE  UNIVERSITY:  RETROSPECTIVES 

►  Mr.  Lee  Henson,  Chief  Agile  Enthusiast,  Davisbase  Consulting 

SPRINT  RETROSPECTIVE  —  FISH  BOWL  EXERCISE 

Participants  discuss  how  it  felt  to  work  in  an  agile  environment.  What 
solutions  and  outcomes  were  identified  by  the  experience  that  can  be  brought 
back  to  the  workplace ? 


STEVEN  HUTCHISON 

Principal  Deputy , 
Developmental  T&E 

Dr.  Hutchison  assumed 
the  duties  as  the  Principal 
Deputy,  Developmental 
Test  and  Evaluation  on  9 
October  2011.  Prior  to  this 
assignment,  Dr.  Hutchison 
served  as  the  Test  and  Eval¬ 
uation  (T &E)  Executive 
for  the  Defense  Informa¬ 
tion  Systems  Agency.  Dr. 
Hutchison  has  also  served 
in  the  office  of  the  Director, 
Operational  Test  and  Evalu¬ 
ation  (DOT&E)  and  the 
Army  Test  and  Evaluation 
Command  (ATEC). 


MARK  KRZYSKO 

Deputy  Director  for 
Enterprise  Information 
&  OSD  Studies 

In  his  position,  Mr.  Krzysko 
directs  data  governance, 
technical  transformation 
and  shared  services  efforts 
to  make  timely,  authorita¬ 
tive  acquisition  information 
available  to  support  over¬ 
sight  of  DoDs  major  pro¬ 
grams;  a  portfolio  totaling 
more  than  $1.6  trillion  of 
investment  funds  over  the 
lifecycle  of  the  programs. 
Mr.  Krzysko  also  oversees 
Federally  Funded  Research 
and  Development  Centers 
and  University  Affiliated 
Research  Centers. 


MICHAEL  KUTCH 

Senior  Engineer,  Joint 
Interoperability  Test 
Command 

Mr.  Michael  Kutch  is  a  pat¬ 
terns  expert  and  a  systems 
engineer  working  for  the 
Joint  Interoperability  Test 
Command  at  Fort  Huachu- 
ca,  Arizona.  Kutch  received 
his  BS  in  electrical  engi¬ 
neering  from  the  Virginia 
Polytechnic  and  State  Uni¬ 
versity  in  1988,  and  his  MS 
in  systems  engineering  from 
the  Southern  Methodist 
University  in  Dallas.  He  is 
currently  working  with  agil¬ 
ity  in  testing  and  organiza¬ 
tions,  and  started  the  JITC 
Agile  Center  of  Excellence. 
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11:45  am -12:00  pm 

12:00  pm  - 1:00  pm 
1:00  pm -2:00  pm 


2:00  pm  -  3:00  pm 

3:00  pm  -  3:15  pm 

3:15  pm -3:30  pm 
3:30  pm  -  4:30  pm 

4:30  pm  -  4:45  pm 
4:45  pm  -  5:00  pm 


AGILE  COACHES  SUMMARY  &  TEAM  EXPERIENCE 

►  Mr.  Michael  Kutch,  Senior  Engineer,  Joint  Interoperability  Test 
Command 

Luncheon  -  Pisces  Ballroom  (15th  Floor) 

GOVERNMENT  POLICY  MAKERS  DISCUSS  DEMOS 

►  Dr.  Steven  Hutchison,  Principal  Deputy,  Developmental  T&E 

►  Mr.  Mark  Krzysko,  Deputy  Director  for  Enterprise  Information  &  OSD 
Studies 

NDAA  SEC  804  PROGRESS 

►  Mr.  Ronald  W.  Pontius,  Director,  C2  Programs  &  Policy,  Office  of  the 
Assistant  Secretary  of  Defense  for  Networks  and  Information  Integration 

WORKSHOP  SUMMARY 

►  Dr.  Steve  Kimmel,  Senior  Vice  President,  Corporate  Development,  Alion 
Science  &  Technology;  NDIA  C4ISR  Division  Chairman 

Afternoon  Break 

OPEN  SPACE  GROUP  WORK 

Networking  group  discussions.  Firming  relationships  for  further  contact. 

OPEN  SPACE  GROUP  0UTBRIEFS 
CLOSING  REMARKS  &  WORKSHOP  SUMMARY 

►  Dr.  Steve  Kimmel,  Senior  Vice  President,  Corporate  Development,  Alion 
Science  &  Technology;  NDIA  C4ISR  Division  Chairman 


STEVE  KIMMEL 

Senior  VP,  Corporate 
Development,  Alion 
Science  &  Technology 

Dr.  Kimmel  directs  Alion’s 
professional  engineering 
and  technology  solutions 
strategic  efforts  associated 
with  homeland  and  defense 
security,  operations  sup¬ 
port,  modeling  and  simu¬ 
lation,  information  and 
wireless  communication, 
technology  integration, 
chemical-biotechnology- 
nuclear  engineering,  human 
terrain  and  intelligence,  ex¬ 
plosive  sciences,  marine  and 
naval  engineering. 


LEE  HENSON 

Chief  Agile  Enthusiast, 
Davishase  Consulting 

Lees  12  years  of  experience 
spans  a  broad  array  of  soft¬ 
ware  production  roles  and 
responsibilities.  He  is  cur¬ 
rently  one  of  just  over  100 
Certified  Scrum  Trainers 
worldwide  and  has  Certified 
well  over  1000  ScrumMas- 
ters  and  Product  Owners. 
Lee  has  worked  hands  on  as 
a  GUI  web  developer,  qual¬ 
ity  assurance  analyst,  auto¬ 
mated  test  engineer,  senior 
product  manager,  senior 
project  manager,  Scrum- 
Master,  agile  coach,  consul¬ 
tant,  and  ADDIE  training 
professional. 


RONALD  PONTIUS 

Director,  C2  Programs 
&  Policy,  OASD  (Nil) 

Mr.  Pontius  is  responsible 
for  the  development  and 
implementation  of  Depart¬ 
ment-wide  C2  policy  and 
direction,  and  executes  the 
responsibilities  of  the  Prin¬ 
cipal  Staff  Assistant  for  C2 
on  behalf  of  the  ASD  (Nil). 
He  is  responsible  for  con¬ 
ducting  program  and  cross¬ 
program  evaluations  of  all 
Service  and  Agency  C2  ac¬ 
tivities  in  areas  of  systems 
engineering,  system  acquisi¬ 
tion,  resource  management, 
and  systems  integration  and 
test. 
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THANK  YOU  TO  OUR  SPONSOR! 

A  VERSIONONE 

Agile  Made  Easier 

VersionOne  is  recognized  by  agile  practitioners  as  the  leader  in  agile  management  tools.  By 
simplifying  the  process  of  planning  and  tracking  agile  software  projects,  we  help  development 
teams  consistently  deliver  software  faster. 

Since  2002,  companies  such  as  Adobe,  Boeing,  Intuit,  Lilly,  Lockheed  Martin,  Oppenheimer, 
Qualcomm,  Sabre  and  Siemens  have  turned  to  VersionOne  to  help  provide  greater  value  to  their 
customers.  Today  more  than  30,000  teams  in  over  170  countries  use  VersionOne’s  agile  project 
management  tools  to  streamline  and  standardize  their  agile  development  efforts. 

Built  specifically  for  Agile  methods,  VersionOne  offers  the  most  feature  rich,  easy  to  use  agile 
management  tool  available.  Using  VersionOne,  all  project  stakeholders  can  work  together  to 
easily  coordinate  project  plans,  priorities  and  progress.  Providing  over  60  reports  and  dash¬ 
boards,  no  other  Agile  management  tool  delivers  more  project  visibility  and  insight. 

Whether  you’re  a  small,  single  agile  team  just  getting  started  with  agile  development  or  a  multi¬ 
team,  global  enterprise,  with  VersionOne  you’ll  get  the  best  tools  in  the  industry  backed  by 
pioneers  in  agile  planning  and  management. 

Agile  Made  Easier  @  VersionOne.com. 
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NDIA  WOULD  LIKE  TO  THANK 
OUR  SPONSOR! 


VERSIONONE* 

Agile  Made  Easier 


DISTRIBUTION  STATEMENT  A:  Approved  for  public  release;  distribution  is  unlimited. 
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Using  Agile  Software  Development  to 
Create  an  Operational  Testing  Tool 

21  Jul  2011 


F.T.  Case 

Jennifer  Ockerman 


(ft.case@ihuapl.edu) 

(jennifer.ockerman@jhuapl.edu) 


APPLIED  PHYSICS  LABORATORY 


Motivation 


■  Rapid  product  development  -  User  needs  definition  to 
concept  development  to  capability  delivery  within  two  years 


■  Classic  Systems  Engineering  method  did  not  seem 
appropriate  for  rapid  product  development 

■  Not  enough  time  to  complete  the  SE  cycle 

■  Not  enough  money  to  fund  the  work  required 


■  Blended  classic  Systems  Engineering  with  an  agile  design 
and  development  approach 

■  Meet  rapid  design  and  development  needs  but 

■  still  deal  with  expectations  of  sponsor  for  traditional  acquisition 
deliverables 


2 


dPL 


The  Blended  Approach 

■  Determined  customer  needs  through  multiple  knowledge  elicitation 
sessions 


■  Created  total  system  requirements  and  top-level  design  upfront  for 
sponsor  approval 


■  Used  agile  methodology  know  as  Scrum  during  system 
development 


■  Each  sprint  had  a  defined  focus  and  product 


Initial  System  Interim 

Requirements  Review  Design  Reviews 


Customer 

Needs 

> 

Requirements 

> 

Design 

Activities 

> 

Development 

> 

Testing 

> 

Delivery 
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Final  System 
Requirements  Review 


Critical  Design 
Review 
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The  System’s  Work  Packages 
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Contents  of  a  Work  Package 


Collection  Status  Utility 

(v50  -  Latest  version,  last  modified  21  days  ago) 

^  (no  tags  set) 

Narrative 

As  a  Configuration  Manager,  I  can  select  a  Collector  Status  utility,  so  that  1  can  monitor  the  real-time  collection  process. 

Acceptance  Criteria 

1.  Verify  that,  the  Video  File  Configurator  allows  fie  Configuration  Manager  to: 

•  set  up  (name)  a  destination  directory  for  video  files. 

*  tag'  (via  file  re-naming  convention?)  files  from  the  video  logger  w/  Test  Event  &  Test  Run. 

*  assign  a  (position/audio  source)  name  to  each  (oftoui)  audio  logger  channels. 

•  modify  current  configuration  (  display  naming  structure) 

*  clear  current  configuration 

2.  Verify  that,  the  Audio  Pile  Configurator  allows  fie  Configuration  Manager  to: 

•  set  up  (name)  a  destination  directory  for  audio  files. 

*  tag  (via  file  re-naming  convention?)  files  from  the  audio  logger  w/  Test  Event  &  Test  Run. 

«  assign  a  (position/audio  soured  name  to  each  (of  16)  audio  logger  channels. 

•  select  pier)  un-wanted  channels  from  being  stored  in  the  OG2ls  file  system  -  QC2IS  ignores  these  channels. 

•  modify  current  configuration 

•  clear  current  configuration 

8.  Verify  that  the  User  is  able  to  see  the  amount  of  storage  space  used  f  bi  00)  vs.  capacity  in  the  drives  fie  RT  collectors  are  writing  to. 

•  up  to  three  (TH|  or  four  (OBJ)  storage  drive  locations  on  the  OC2IS  system  should  be  able  to  be  monitored. 

*  this  status  (scrolling  bar  graph?)  should  change  colors 

*  at  68%  unused  capacity  (YELLOW) 

•  at  25%  unused  capacity  (RED) 

4.  Verify  that  there  is  an  Network  Time  Protocol  (NTP)  Server  area  fiat  allows  fra  Configuration  Manager  to: 

•  see  the  current  NTP  Server  time  being  broadcast  to  all  machines, 

*  allows  the  Configuration  Manager  to  call  the  NTP  server  configuration  window  (via  http*  script?)* 

*  see  the  status  of  NTP  polling  of  all  machines  on  the  OTS  network. 

§L  Verify  that  the  NTP  Server  polls  all  machines  on  the  OTS  network  every  1 8  seconds  and  chaecks  to  see  that 

*  time  is  accurate  to  the  second  tor  the  last  5  poll  cycles  -  GREEN 

•  there  is  a  time  error  >  isecfor  up  to  two  of  fra  last  s  polling  cycles  -  YELLOW 

*  there  is  a  time  error  >  2sec  or  an  error  >1  sec  for  3  or  more  of  the  last  5  polling  cycles. 


Mockup 


Whan  th*  application  It  (Ini  opanad  tha  cor*ig»ratlon  manager  matt  aaiact  a  tat*  amnt/nm  and  than 
avaryttang  that  happant  unM  tha  application  It  clatad  raportt  an  or  Impacts  that  last  mnt/run 


I  Inin  laid  SEgrlpp 


WKlt 
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API 


Satisfied  sponsor  need  for  requirement  and  configuration  item 
documentation  in  early  stages 

Established  a  collaborative  development  lab  to  maintain  contact  with 
customer/sponsor/remote  team  members  during  project  execution 

■  Telecon  lines,  web-based  collaboration  tools 

Self-synchronization  of  the  team  -  shift  roles  and  activities  to  meet 
current  needs 

■  Example:  Hardware  architect  also  tests  code 

Product  owner,  developers  and  tester  in  same  lab  during  same 
hours  to  facilitate  communication  beyond  daily  update  meetings 

■  All  but  one  of  the  team  members  are  part-time  -  work  afternoon 
together  in  lab. 

Adaptable  to  new  customer  needs  and  suggestions  during 
development 


AGILE  PROCESSES 

EXAMPLE 


Integrity  -  Service  -  Excellence 


equirements:  Where  do  they  come 

from? 


Users 

•Helpdesk 
•Field  Events 
•Conference  inputs 
•PUC 

•Local  Training 


Program  Manager 

•Paying  customer  inputs 
(MAJCOMs) 

•PMR 
•Wild  hairs 


ops  # 


Big  R/Little  r 


VersionOne  Request 
•Gross  or  refined,  does 
not  matter-just  get  it  in 


none 


PEXTeam:  Internal 
•  Devs/S  MEs/Testers 
•Architecture  Committee 
•Etc. 


Wim  We  nee 


equirements:  Where  do  they  come 

from? 


POT  Team 

•Release  Planning 
converted  to  stories 
•Execution 
generates  stories 


Dev  Team 

•Revealed  during  execution 
•Revealed  during  iteration  test 


:t 


Little  r 
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POT 


VersionOne  Stories 
•Not  "just  get  it  in" 
•Specific  to  support 
release  effort 
•No  wild  hairs 


" Welcome  changing  requirements ,  even  late  in 
development.  Agile  processes  harness  change  for 
the  warfighter's  competitive  advantage." 


POT  team  has  primary  responsibility  to  keen  effort  aligned  with  priorities  during  execution 

Integrity  -  Service  -  Excellence 


Requirements  Process  timeline: 
How  we  Filter/Prioritize/Refine 


OPS: 

•Consolidates  VI  requests 
•Generates  2YR  plan  inputs 
•Prepare  recommendations  for  PM 

POT: 

•Creates  ROM  estimates 
•Generates  2YR  plan  inputs/refinements 
•Prepares  recommendations  for  PM 
•Draft  Roadmap 


Pre-release 

Planning 


Release  Planning 


Release  Execution 


Post¬ 

release 


Program  Manger:  "Business  people  and  developers  must  work 

•Coordinates  with  Stakeholders  together  daily  throughout  the  project." 

•Conducts  PMR 

•Obtains  Stakeholder  Concurrence 

•Approves  Roadmap  Collaboration 


Integrity  -  Service  -  Excellence 


Requirements  Process  timeline 
How  we  Filter/Prioritize/Refine 


POT:  (SMEsand  Developers) 

•Based  on  Approved  Roadmap 
•Decomposes  Requirements  by  Function 
(Vertical  Slice) 

•Prepares  for  Release  Planning  Meeting 
•Requirements  and  Design  meetings 
•Acceptance  Criteria 


POT  conducts  Release 
Planning  Meetings 
•Aligns  Team  workload  with 
strategic  intent 
•Establishes  initial  workflow 
for  the  release  with  the 
Teams7  input  through  Scrum 
Masters  (learning  point) 


re-reiease 


Planning 


Release  Planning 


yj 


Development  Teams 

•Provides  Inputs  through  design  meetings 
•Creates  SketchFlows 


Release  Execution 


release 


OPS: 

•Refines  understanding 
of  requirements  in 


response  to  POT 

■Iterative  ‘3  e  r  v  i  c  e  -  E  x  c  e  1 1  e  n  c  e 


Requirements  Process  timeline: 
How  we  Filter/Prioritize/Refine 


r>R>R 


Development  Teams 

•Executes  delivery  of  functionality 

•Organic  testers  and  SMEs  expose  additional  T 

•Defers  to  POT  for  requirement  clarification  &  sequencing 


No  d@liv@ry  @xf@nsions 


re-reiease 
Planning 


Release  Planning 


Release  Execution 


release 


POT  focus  shifts  to 

subsequent  release's  "Our  highest  priority  is  to  satisfy  the  warfighter 

•pre_re|ease  planning  through  early  and  continuous  delivery 

of  valuable  software  " 


Integrity  -  Service  -  Excellence 


timeline: 

How  do  we  know  we  have  met 

them? 

Development  Teams 

•Iteration  Demonstrations 
•Iteration  Retrospectives 
•Were  Acceptance  Criteria  adequately  defined? 

• Existing  Requirements  are  validated  at  Iteration  Test,  not  Systems  Test 


OPS  Team 

•Conducts  Systems  Test  against  validated  acceptance  criteria 
•Uses  Regression  Tests  based  on  the  actual  ATs  created  for  each 
slice  of  functionality 

•Systems  Test  is  part  of  Release  Execution  and  ends  on  time 


Pre-release 

Planning 

Release  Planning 

r - i 

Release  Execution 

Post-  1 

release 

Government  Customer 

•  Working  groups 

•  Warfighter  comes  to  PEX  office 
•PEX  Users  Conference 
•High  Level  Verification 
•Beta  Test  Sight  for  Warfighter 
•What  does  the  PM  think? 


Requirements  Process  timeline 


Development  Teams 

•Release  Retrospective 


OPS  Team 

•Release  Retrospective 
•Installation  and  Training  lead  to 


How  do  we  know 
we  have  met 
them? 

new  requests 


Independent  Testing  Events 

•46  TS  risk  mitigation 
•SSI  DT&E/FDE 
•Working  with  AFOTEC 


Pre-release 

Planning 

Release  Planning 

Release  Execution 

Post-  " 
release 

Release  Candidate  sites  (HLV) 

•Feedback 

•New  entered  into  VersionOne  as 
Requests 


Integrity  -  Service  -  Excellence 


onePEX  Design  focus: 


•  Compatibility/Consistency  across  PEX:  standards 

•  Usability  for  its  target  audience:  the  warfighter 

•  Intuitive  work  flow 


Why?  Why?  Why? 


Pre-release 

Planning 


Release  Planning 


Release  Execution 


Post¬ 

release 


To  achieve  the  desired  collaboration  we  emphasize  use  of 
the  tool:  SketchFlow 


Integrity  -  Service  -  Excellence 


Design  flow 


onePEX  Design:  (Developers) 

•Keeps  SharePoint  current 

•Meets  regularly  to  ensure  consistency  across 

the  effort  (Thursday  after  Planning  Meeting) 

•Write  defects  for  prioritization  and 

sequencing 


Developers: 

•Resolve  differences  between  SketchFlow 
and  Story  Acceptance  Criteria  with  POT 
•Resolve  differences  between  Sketchflow 
and  Acceptance  Tests  with  onePEX  Design 
•Tasks  are  design 

•Iteration  demonstrations  are  not  design 
meetings  but  they  do  expose  your  work 


Pre-release 

Planning 


Release  Planning 


Release  Execution 


Post¬ 

release 


Embedded  SMEs: 

•New  requirements  are  provided  to  POT  "Simplicity-the  art  of  maximizing 

•Design  issues  are  provided  to  onePEX  Design  the  amount  of  work  not  done-is 

for  resolution  essential " 


Integrity  -  Service  -  Excellence 


Development 


"Working  software  is  the  primary  measure  of  progress." 

Some  fundamentals  about  Agile: 


post  work ,  visibly 
stop  starting,  start  finishing 
done  is  done,  begin 


Pre-release 

Planning 


Release  Planning 


Release  Execution 


Post¬ 

release 


Integrity  -  Service  -  Excellence 


Development 


"Working  software  is  the  primary  measure  of  progress." 


Development  Teams 

•Participates  in  Requirements  and  Design  meetings  in 

accordance  with  the  roadmap 

•Shapes  estimates  in  the  Release  Planning  Meeting 

•Help  establish  the  delivery  sequence 

•Review  Acceptance  Criteria,  SketchFlow  and 

Acceptance  Tests  to  prepare  for  writing  tasks 


"The  best  architectures ,  requirements ,  and  designs 
emerge  from  self-organizing  teams  " 


"Continuous  attention  to  technical  excellence 
and  good  design  enhances  agility." 


Integrity  -  Service  -  Excellence 


Development 


"Working  software  is  the  primary  measure  of  progress." 


No  dolivtry  oxfonsions 


PEX  Team 

It's  show  time 


re-reiease 
Planning 


Release  Planning 


Release  Execution 


release 


"Deliver  working  software  frequently,  from  a 
couple  of  weeks  to  a  couple  of  months,  with  a 
preference  to  the  shorter  timescale  " 


Integrity  -  Service  -  Excellence 


Integrity  -  Service  -  Excellence 


PATRIOT  EXCALIBUR 

(PEX) 


Kelly  Goshorn 
ESC/HSGG 


Date:  14  Nov  11 


OUTLINE 


The  purpose  of  this  panel  is  to  share  PM 
experiences  executing  Agile  development  in 
DoD  -  to  “help  others  to  do  it” 

•  Why  did  PEX  go  Agile? 

•  External  Customers:  Manage  Stakeholder 
Expectations 

•  Contract 

•  Changing  Practices 


Integrity  -  Service  -  Excellence 


Why  Agile? 


•  2003 

-  Traditional  waterfall:  documentation  nightmare 

-  Fielded  our  first  version,  came  up  short 

-  Late  in  delivery 

-  Funding  at  risk 

•  We  had  to  do  something  different,  or  die 


Message:  Crisis  facilitates  change,  encourages 
“risky”  moves 

V _ 


Integrity  -  Service  -  Excellence 


Manage  Stakeholder  Expectations 

•  Main  focus  of  PM’s  energy 

•  Stakeholders  include  the  users:  meet  the 
warfighter’s  needs 

•  Involve  them: 

*  Membership  on  the  team--SMEs 

*  In  defining  the  requirements 

*  Establishing  priorities 

*  PUC 

*  Working  Groups 

*  PMR 

\ 

Message:  People  support  that  which  they  help  to  create 

V _ J 


Integrity  -  Service  -  Excellence 


Program  Contract 


•  Manage  the  Program,  not  the  Contract 

•  Contract  is  Cost  Plus  Award  Fee,  Software 
Engineering  Support  (4  different  contracts) 

*  If  you  specify  a  function  in  a  contract,  that’s  what 
you  get 

(whether  you  need  it  or  not) 

*  Requires  a  solid  process  to  add  detail  to  the  “Big 

R” _ 

Message:  You  are  not  going  to  establish  the 
collaborative  relationship  with  your  contractor 
required  to  deliver  the  goods  by  locking 
^  requirements  delivery  into  a  contract _ 


Integrity  -  Service  -  Excellence 


Changing  Practices 


•  2003-took  the  hit  on  schedule,  instituted  Agile 
*  Small  team  comprised  of  developers  and  SME 


•  2005-growth  led  to  expansion  of  team 

*  Operations  Subject  Matter  Experts  led  growth 

•  2008-Broke  team  into  smaller  teams  to  align 
assets  with  objectives 

•  2011 -Applied  lessons  on  large  (100+  people) 
organization  employment  of  Agile,  reorganized 
to  adopt  appropriate  Agile  practices 

Message:  Practices  change  depending  on  environment,  technology 
and  product.  Values  and  Principles  are  constant 

v 


Integrity  -  Service  -  Excellence 


Manifesto  for  Agile  Software  Development 


We  are  uncovering  better  ways  of  developing 
software  by  doing  it  and  helping  others  do  it. 
Through  this  work  we  have  come  to  value: 

Individuals  and  interactions  over  processes  and  tools 
Working  software  over  comprehensive  documentation 
Customer  collaboration  over  contract  negotiation 
Responding  to  change  over  following  a  plan 

That  is,  while  there  is  value  in  the  items  on 
the  right,  we  value  the  items  on  the  left  more. 


The  Coming  American _ _ 

Bring  me  men  to  match  my  mountains; 
Bring  me  men  to  match  my  plains, 
--Men  with  empires  in  their  purpose, 
And  new  eras  in  their  brains. 

Sam  Walter  Foss,  1894 


Yes,  Sir. 

No,  Sir. 

Sir,  may  I  ask  a  question? 

Sir,  may  I  make  a  statement? 


No  excuse,  Sir! 


...to  change  the  world 

Working  software  is  the  primary 
measure  of  progress. 


